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^ An experiment was performed to show that' infants 
perceive auditory and visual stimuli within a common, space and that 
they perceive the sound as an attribute of the visual object. 
Subjects were 22 infants aged 3 to 5 months. Each infant was. 
presented with a toy that moved in a small arc from side to side of a 
small window at the rate of one arc per second. The back-and- fourth 
movements occurred in synchrony with chime. ^ The sound was^ wired 
either to come from the ffont window where the toy was located or 
from 90 degrees to the side behind a cloth. Two dependent measures 
^^ere recorded: (1) limb and body movement, and (2) visual orientation 
€o the left or right."* Infants showed^ncreases • in agita;ted. limb and 
body movement when presented with the chime spatially dislocated from 
its temporally synchronous visual source (the toy). Agitatibn was not 
shown when the chime and the toy were presented together in spao^^. 
Infants who 'demonstrated adequate processing of the auditory 
information through accurate orientation behavior were laost likely to 
show increased agitation. (Author/BET) ' * 
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The research that I shall present today is aimed at exploring the 

infant's experience of an event which is composed of both auditory and visual 

features. The origins of man's ability to organize information from seve- 

ral quite different «Freceptor systems into a upitfied perceptual world of ob- 

Jects an^d events, has long interested students of human nature. We do not 

expedience visual, auditory, and tactile sensations as disconnected inputs. 

Instead we experience organized objects or events which have multimodal 

attributes, such as a dog who is soft and black and barks or* in Bishop Ber- 

o 

keley's classic examptle, a solid, sjiiny, clattering coach. 

However, until recently psychologists have predominantly accepted 
Bfshop Berkeley's (1709) view that the various perceptual modalities are 

structured initially as separate systems (e.g. Birch and Lefford, 1967; Pick, 

* ' ■ • • ■ . 

Pick, and Klein, 1967). Focusing his analysis particularly on spatial cues, 
he proposed that it is only through repeated experience of an association be- 
tween crossmodal inputs that information comes to be correlated across^ modes. . , 

Piaget.is the developmental theorist^ost identified with this 
viewpoint (Piaget, 1952; Plaget, 1954). He der,Sr/ibes early perceptual devel- 
opment as a prbcesG involving the gradual reciprocal assimilation 0f ^pal^ial 
schemas which are initially modality specific. ^ . • 

Others have disputed this theory and pro^^osed that structures, allow- 
Ing for the spatial coordination of information from different modalities are 
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a pa^t of the hiiman^ perceptual system. Among developmentalists^ Bower <1972) 
* ■ ' -i ' • * 

has presented evidence that within two weeks aftet birth iiifants expect tac- 

tual cues ,as to size and^dis^tance of objects to covary with visual size and 

distance cues, firuner and Koslowski (1972) have presented similar findings ^ 

with pre-reaching infants. 



Regarding. coordination of visual And auditory information, recent 
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neurophysiological research with lower mammals has shown that bimodal cells ^xist 

in the visual cortex of the cat which serve to correlate the spatial locations 

^ / t • ■ 

of concurrent aud^^ory and visual inputs (Murata, Cramer and Bach-Y-Rita, 

\ 

1965; Fishman and Mii:hael, 1973). However, for 'the human infant, no conclu- 

slve evidence of early integration^ of auditory and^ visual spatial informa- 

* ■ • , ' - 

tion has been provided to date. Even normative data on the development of 

spatial orientati45n behavior to auditory cues is meager md inconsistent. 

/Visual orientation to auditory cues is not sufficient in i^^lf , however, 

I 

. ■ • ■> ■• * 

to^ demonstrate integration of visual and ajuditorjr perceptual information* 

Orientation of 'the head may 'be guided- primarily Sy an auditory centering 
mechanism rather than a visual or audio-visual mechanism. Secondly, the * 
presence of an oculomotgr response to sound can, at most. Indicate tliat 
audftory spatial infoirmatlon can be transduced into general directional 
Information- to- the"^ oculo-motor system. Such a response tells us nothing, 
however, about t\ie organization of the visual and auditory perceptual pro- 
cessing ^^ich follows thd motor response; Recent studies of concurrent . ^\ 
•udltory and-^vlsual processihg' in infants by Udnnee (1973), Carpenter (1973) 

I(n4 Cohtn (1973) have not. directly addressed tfie issue of Integratiqn of 
\'-.V-^^ • . • ■ . 

0p#t\al Information. 

Aronsoii and feoaenbloom (1971) did directly explore the issue of • 
. .... ' ^ " ■ -. 

8patldl,\Untegration^ by tiomparing } the 4- tov^O-week-old infant "^s response 
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o ■ , f *. 

to spatially coincident and spatially separated presentations of mother's 
face and voice. They concluded that infants expect mother's face and voice 
to occur together in space and become disturbed if this expectation is vio- 
lated. This interpretation is not clearly supported, however, due to the use 

of an unvalidated measure of disturbance, tongue protrusion, aiid the lack 

r 

of a counterbalanced presentation order. Thus no comparison is presented in 
^that study which distinguishes a response to the dislocation k>f sig^gpiand 
.sound from a re^sponse to any change , in the stimulus configuration. In ad- 
dition, as Bower (1974)- has pointed out, agitated behavior to the dislocation 
inay have resulted not from a violation of expectation but from response or ^ . 
processing difficulty generated by tjie increased spatial complexity of the 
stimulus. Finally, McGurk and Lewis (1974) have recently reported their 
failure to replicate the Aronson and Rosenbloom findings with infants of 
1 to 7 months of age when using an improved methodological patadigm. ^ 

The study I will present today had two aims: first, to replicate 
the essential features of the Aronson and Rosenbloom study^ with an improved 
methodology; second, to investigate Bower's alternative interpretation of 
response conflict by also monitoring the infant's auditory orientation 
responses. y. \ ' ^ [ 

METHOD ' ! 

Twenty-two infants participated in the study, six at 3 months of 

age, eight at 4 months and>^ight at 5 months of age. 



Request Figure One here 



This Is a schematic drawing of the experimental situation. 'Each infant was 
presented with a toy that moved in a small arc from side to side of the^ wlii- 
dow at the rate of one. arc per second. '^The })adk-anH^£orth movements occurred 
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In synchrony wijh a chime. The sound was wired to Issue either from the 
front wln(low where the toy was located or from 90 degrees to. the side from 
behind a cloth, S^nd pressure levels were 70 decibels from both locations. 
Each subject observed two ' continuous episodes each of 80 seconds duration, 
one in which sound and object were collocated at the front window, one in 
which sound was dislocated 90 degrees to the right, jjjfie order of presentation 
of these two episodes was count^i^lanced across^^ubjects. In order th em- 
phasize the temporal relationship between sound occurrence and object move- 
ment, object movement with sound occurred during each episode in 8 5-secon<^ 
Intervals alternating' with 8 5-second intervals of no sound and no movement. 
Sound location change occurred during the eighth silent interval. In order 
to determine whether stimulus familiarity is am important factor in the^ in- 
•ant's reaction to the spatial dislocation, the amount of prior exposure to 
the sounding object was also varied. Half of the infants viewed the counter- 
^ balanced best presentation after no previous exposure to the object; the 
other half viewed it after 160 seconds previous exposure to the sounding ob- 
Ject. , • 

Two dependent measures were recorded. 'The first was limb and body 
movement; shown by Haith, Kessen and Collins (1969) to be an infant behavioral 
r^spopse to an unexpected rule violation. The second behavior monitored 
was visual orientation to left or right as indexed by a head turn of approxi- 
jnately 30 degrees or more. 

HESULTS" . ^ 

Analysis of variance qf the movement and the orientation measures 



yielded the following results. ^ 



^Sincft wl thin-group variances of both the movement and looking scores were 
found to be a linear function of group means, all scores were)^ subjected to 
a 8quarf-root transformation before data analysis. 
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Request Table One here ^ 

Limb ^nd body movement of the^^Jij^ants was more pronounced during the dls- 

( ■ . / 

location episode than during t^e collocated episode, regardless of the order 

df presentation (F(l, 10)^5. 385f 'p<'0'43). This effect was most evi^nt 

during the second 40~second period of the episode (F(1,10)*5.619; p<.040). 

Neither the age of the infants nor the amount of prior exposure to the soun- 

* 

ding object reliably affected the amount of limb and body movement. 



Request Figure two here 



With regard to the orientation data. Infants looked away f^rom the 
forward object very infrequently. However, looking away to the rlghjt oc- 

curred more often during the first 40 seconds, that the sound Issued from 

■ " • '■ . 

■** 

the right than during the comparable period when the sound Issued from the 
front. ^Thls effect was true only for the 4- and 5-month old Infants, however. 
The 3-month-old Infants, in contrast, looked to th^ right less during the 
dislocation episode (Age X Episode: F(2,10)=6.069; p<. 019). Left looking 
was not related to any of the exp^lmental factors. McGurk and Lewis, in 
their study, also reported accurate auditory orientation, to both right and 
left, by their 4- and 7-month-old subjects but not by their 1-month-old 
subjects, , • ; . . . 

Pea analysls^f the relationship between looking behavior and move- 
neilt Increase on a subject-by-subject basis revealed that the infants who 
oriented at least once toward the dislocated sound were the Infants most 
' 4.1kely to «how an Increase in agitated movement during the dislocation epi- 
iodl^o (Fisher* 8 Exact P*.026). Looking to the left', however, was nqt related 
to movement Increase (Fisher's Exact P-.490) (Sec Table 2). 

ERIC 0 0M6 . , 



\. ' • 

DISCUSSION \ ^ ' • ' 

\ ■ • 0 

What conclusions can. we draw from this pattern of results? The 
major finding was that agitated limb^and tody movement was more pronounced 
during the dislocation episode. This increased ^^gitation cannot be attri- 
buted to stimulus change per/se, since change of^^^e 30und from the disloca- 
ted to the normal position produced quieting. ^ . 

Could the agitated behavior have resulted from a response conflict 
due 5:0 two concurrents ^patial sources of input? Presumably, if this were the 
case, one would expect agitation to be most pronounced in the younger or the 
non-orienting subjects and to decline as response capabilities improve. This 
was not the case. Instead, it is the more mature, accurately orienting in- 
fants- who show the most reliable increases in agitated movement. Response 
conflict might still be a credible factor if it could be shown that the agi- ° 
tatlon preceded auditory orientation over the course of the experimental 
trials. This was not the case. As mentioned earlier, right, looking occurred 
most reliably^-tftlring th4 first AO-secom period while movement increase ^ 



occurred most r/liably during the second 40-second period. It should also 
be noted that when both episodes are considered left looking occurred as 
frequently as right looking yet left looking shows no relationship to move- 
ment Incteade. Response conflict , alone then seems unlikely, to be a deter- 
mining jEac tor causing the movement increases. ^ 

, » \ 

Could processing difficulty rather than response difficulty have ^ 
^ been a major factor in the movement increases? The points that I have just 

mentioned also weigh against the processing difficulty hypothesis. The in- 
• ffttits who demonstrate their ability to adequately process the bimodal spatial 

Information are the same subjects who show the increases in agitated movement. 

^ ' J . . p 



As mentioned, the 3-month-old infants did not orient 
toward the dislocated sound. Thus, one cannot conclude unequivocably 
that the 3-month~old .subjects were coordinating the auditory and 
visual information. Response conflict, howeverj can be ruled out 
as an explanation for the pattern of behavior of the three-month- 
old infants. Post-session responses to a lat^ralized sound, in 
the absence of a competing visual stimulus, were collected from a 
subsample of 12 infants from the present study and from a larger ^ 
sample of 90 infants aged 11 to 16 weeks ofjaL$e. Only 10% of the 
infants 11-13 weeks of age oriented head and^eyes toward the audi- 
tory stimulus, while 81% of infants" aged 16 weeks did so ^ 
2 ' 

(Xt « 19.196, p < .001). Thus, auditory orientation ig^ not an avail- 
able competing response for the younger infants. ^ ' / 

Slower processing speed, on the other hand, may have 
influenced the younger infants' responses though apparently with 
the effect of diminishing, rather than enhancing, the movement 
response* Of the subgroup of subjects given the orientation 
post-test, those who failed to orient on the post-test also 
failed to orient within the experimental session. During the 
experimental episode^, those non-orienting infants showed marked • 
quieting before beginning to activate motorieally, both^^fCT^e 
onset of the session and at the change in sound locacion, 
whether the change occurred toward or away froisi the visual stimulus. 
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In contrast, orienting subjects quieted only to the change toward 
the stimulus and activated to the change away from the stimulus 
(Orientation Group X Change Score: F(2,16) = 6.447, p < .009). 
This tendency of the less mature infants to quieten to stimulus 
change suggests that quieting rather than activation is the most 
likely behavioral reaction to a difficult processing task. Though 
the non-orienting Infants did' register the change, they may have 
had more difficulty proce^ing the stimulus information in the 

time allowed. The data ta^cen as a whole suggest that the younger 

'I 

Infants were processing the visual and auditory information but 
more slowly than the older Infants. Data from longer experimental 
presentations^ are needed to evaluate this interpretation. 

Some further data support the interpretation that the 
agitation of the Infants results from a rule violation rather 
than from the physical complexity of the dislocation episode per se. 
An additional group of 8 4-month old Infants were presented with 
the sane procedure with two modifications: (i) no sound was ^ 
lu^ltlally assdelated with the forward* object and (11) when the \ 
sound occurred 90 degrees to the right there was also an object 
on the right. The spatio-temporal information presented wd^ ^ 
thus equally complex in the QO^degree condition, but no rule 
vldlatlon was Involved. Under these clrcumstances.-<rhe infants 
showed a non-significant tendency to decrease, rather than Increase 
limb and body moveme'it (F(l,6) 3.569; if < .108). * 

9 0 009 ■ ^ 
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The disparity between these results and the lack of positive 
findings by McGijrk andaewis is most likely due to the different dependent 
variables monitored as ihdices of infant disquiet. The two significant 
effects obtained in their study, accurated auditory orientation from 4 
months of age and decreased smiling after the dislocation 'at 4 months of 
age, support jthe present conclusions. 

The pattern of the data, then, supports the conclusion thats (1) 
the spatio-temporal information presented is being adequately processed by 
the older infants and that (11) the agitated movement is occurring in re- 
sponse to this information. Thus, the Infants appear to interpret the 

t e 

dislocation according to the following rule: Sounds which occur in tem- 
poral synchrony with the movements of a visual object are features of that 
objec\ and Aould move in space with that object. The unfamiliar ity of the 
objects presented ^d the lack df a differential effect of prior exposure 
suggest that a coordinated auditory-^visual percept does not develop through* 
exteiisive prior experience but is the direct product of a prestructured 
^ jiulfilmodal perceptual syjstdm. On the basis of the present data it is also 
, proposed that the perceptual system is prestructured to attribute a sound 
to a particular visual source on the basis pf temporal synchrony in the 
occurrence of auditory and visual 6hange. From very early, then, visual 
and auditory features are not preceivedy according to spatially separate, 
ao^allty-speclfic schemas. Instead, visual and auditory features are per- 
ceived ^thin a common spatio^-temporal framework as aspects of the same 
event* The Infant experiences a solii, €liree-dlmensional object which 
chiies* 
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Tables X. • Influence^ 9f auditory-visual dislocation on infants' 
limb and b<^y movements . . Sound issues from the fjont in 'fepipode A 
from the right in' episode B. Each score is the mean number of . 
millimeters of recdrding pen deflecti^on produced per 5-second 
interval. " * : " " 
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standard - Dislocation 

SOUND PLACEMENT ' 

FIGURE Z. Right looking during Period One as a function of 
sound placement an(| age: Study I 
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